A Few Days of A Robot's Life in the Human's World: Toward Incremental Individual Recognition

PhD thesisThis thesis presents an integrated framework and implementation for Mertz, an expressive robotic creature for exploring the task of face recognition through natural interaction in an incremental and unsupervised fashion. The goal of this thesis is to advance toward a framework which would allow robots to incrementally ``get to know'' a set of familiar individuals in a natural and extendable way. This thesis is motivated by the increasingly popular goal of integrating robots in the home. In order to be effective in human-centric tasks, the robots must be able to not only recognize each family member, but also to learn about the roles of various people in the household.In this thesis, we focus on two particular limitations of the current technology. Firstly, most of face recognition research concentrate on the supervised classification problem. Currently, one of the biggest problems in face recognition is how to generalize the system to be able to recognize new test data that vary from the training data. Thus, until this problem is solved completely, the existing supervised approaches may require multiple manual introduction and labelling sessions to include training data with enough variations. Secondly, there is typically a large gap between research prototypes and commercial products, largely due to lack of robustness and scalability to different environmental settings.In this thesis, we propose an unsupervised approach which wouldallow for a more adaptive system which can incrementally update thetraining set with more recent data or new individuals over time.Moreover, it gives the robots a more natural {\em socialrecognition} mechanism to learn not only to recognize each person'sappearance, but also to remember some relevant contextualinformation that the robot observed during previous interactionsessions. Therefore, this thesis focuses on integrating anunsupervised and incremental face recognition system within aphysical robot which interfaces directly with humans through naturalsocial interaction. The robot autonomously detects, tracks, andsegments face images during these interactions and automaticallygenerates a training set for its face recognition system. Moreover,in order to motivate robust solutions and address scalabilityissues, we chose to put the robot, Mertz, in unstructured publicenvironments to interact with naive passersby, instead of with onlythe researchers within the laboratory environment.While an unsupervised and incremental face recognition system is acrucial element toward our target goal, it is only a part of thestory. A face recognition system typically receives eitherpre-recorded face images or a streaming video from a static camera.As illustrated an ACLU review of a commercial face recognitioninstallation, a security application which interfaces with thelatter is already very challenging. In this case, our target goalis a robot that can recognize people in a home setting. Theinterface between robots and humans is even more dynamic. Both therobots and the humans move around.We present the robot implementation and its unsupervised incremental face recognition framework. We describe analgorithm for clustering local features extracted from a large set of automatically generated face data. We demonstrate the robot's capabilities and limitations in a series of experiments at a public lobby. In a final experiment, the robot interacted with a few hundred individuals in an eight day period and generated a training set of over a hundred thousand face images. We evaluate the clustering algorithm performance across a range of parameters on this automatically generated training data and also the Honda-UCSD video face database. Lastly, we present some recognition results using the self-labelled clusters

Recognizing and Remembering Individuals: Online and Unsupervised Face Recognition for Humanoid Robot

Individual recognition is a widely reported phenomenon in the animal world, where it contributes to successful maternal interaction, parental care, group breeding, cooperation, mate choice, etc. This work addresses the question of how one may implement such social competence in a humanoid robot. We argue that the robot must be able to recognize people and learn about their various characteristics through embodied social interaction and thus proposed an initial implementation of an online and unsupervised face recognition system for Kismet, our sociable robotic platform. We show how specific features of this particular application drove our decision and implementation process, challenged by the difficulty of the face recognition problem, which has so far been explored in the supervised manner. Experimental results are reported to illustrate what was solved and lessons learned from the current implementation

Out in the World: What Did The Robot Hear And See?

In this paper, we present preliminary results obtaine

Online and Unsupervised Face Recognition for Humanoid Robot: Toward Relationship with People

The Problem: The ability to distinguish among different individuals is crucial for all social animals. Observations of behavior within dolphin male coalitions indicate that dolphins are able to not only recognize friends, but also keep track of their previous behavior to predict future action. This work addresses the questions of how one may implement such social competence in a humanoid robot. Information about various people’s identities (appearance, behavior, etc) that we acquire through our daily social experience is so rich and complex that manually encoding them into a database for the robot to memorize is very limiting. In this project, we focus on extending current face technology to allow the robot to opportunistically learn about individuals and their characteristics in an online and unsupervised manner through embodied social interaction. Motivation: The notion of people as distinct individuals plays averyimportant role in our daily social life. If a robot has the ability to recognize and remember people it interacts with, it will be able to learn about characteristics of each individual and treat them uniquely as individuals. This leads to complex social behavior, such as cooperation, dislike, loyalty, and affection. As proposed by [3], if robots have long-term contact with humans, it may be desirable to have them develop individual relationships, which is exactly the aftermath of this social dynamic. Moreover, the ability to distinguish among people allows therobot to build toward more complex social competencies where the idea of people as distinct individuals is crucial, including theory of mind and social referencing

An exceptional handling service for the contract net protocol family

Thesis (M.Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1999.Includes bibliographical references (leaves 61-63).Autonomous agents are systems that inhabit complex and dynamic environments. Attaining robust behavior in such complex conditions is a key challenge for agent-based systems. This challenge is intensified in multi-agent settings, where a diverse set of agents communicate and coordinate with each other, resulting in non-deterministic system behaviors. The standard approach to this problem relies on the notion that each agent should be equipped as much as possible in order to survive on their own in its environment, leading to several serious limitations. Dellarocas and Klein (1999) propose an alternative approach, using a generic exception handling service for providing adaptability by detecting and handling domain independent exceptions. This thesis presents an evaluation and prototype implementation of this approach using a contract net setting. The feasibility of the approach is assessed, involving identification of contract net related domain independent exceptions and their handling strategies as well as an investigation of the minimum base level interfaces required from each agent. Experimental results showing a scenario in which agents relies on the exception handling service for dealing with agent death exception are demonstrated. Lastly, a thorough analysis and comparative study of the shared service approach versus other related works are presented.by Lijin Aryananda.M.Eng